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INTRODUCTION 


At  present  the  naval  aviation  training  flight  syllabus  consists  of  four 
phases:  primary,  basic,  advanced,  and  the  replacement  air  group  (RAG).  The 
first  three  of  these  are  considered  the  undergraduate  level  of  training .  Upon 
completion  of  the  advanced  phase,  the  student  pilot  is  awarded  his  wings  and  is 
designated  a  naval  aviator .  The  RAG  is  the  postgraduate  phase  of  training 
wherein  the  student  pilot  is  first  introduced  to  the  operations  of  a  fleet-type  air¬ 
craft  .  Upon  completion  of  this  phasfe ,  he  is  assigned  to  a  fleet  squadron . 


Most  research  in  naval  aviation  has  been  devoted  to  the  isolation  of 
skills  and  the  prediction  of  student  pilot  success  at  the  undergraduate  levels  of 
training.  The  success  of  such  efforts  is  well  documented.  Nevertheless,  the 
fact  remains  that  a  naval  aviator  is  trained  to  become  an  integral  part  of  an  opera¬ 
tional  fleet  squadron .  The  ultimate  criterion  for  pilot  performance  must  neces¬ 
sarily  result  from  the  manner  in  which  he  fulfills  the  mission  objectives  for  his 
particular  aircraft.  The  development  of  a  criterion  for  fleet  performance 
remains  a  pressing  problem— one  which  is  the  focus  of  the  present  research 
effort. 


Several  attempts  have  been  made  to  develop  criterion  measures  for  cer¬ 
tain  fleet  operations .  Using  an  index  derived  from  the  arrestment  wire 
engaged,  Brictson,  Burger,  and  Kennedy  (1)  reported  a  regression  model  in 
which  altitude  error  and  sink  rate  were  the  best  predictors  of  the  quality  of 
night  carrier  landings.  In  a  later  study,  Brictson,  Burger,  and  Gallagher  (2) 
reported  that  certain  selection  test  scores  and  stage  grades  during  training  were 
significantly  related  to  the  quality  of  carrier  landings  as  measured  by  a  number 
of  objective  indices .  While  proficiency  in  carrier  landings  lends  itself  to  some 
degree  of  objectivity,  such  is  simply  not  the  case  for  other  operations.  Conse¬ 
quently,  the  researcher  must  rely  upon  subjective  measures  of  performance. 
Despite  the  inherent  problems  associated  with  the  use  of  ratings,  they  are  often 
the  only  measures  which  are  available .  Interestingly  enough ,  the  study  of 
Brictson  et  al  (1)  reports  measures  derived  from  the  subjective  estimates  of  the 
LSO  (landing  signal  officer)  to  be  highly  correlated  with  objective  estimates 
derived  from  a  weighted  combination  of  wave-offs ,  bolters ,  and  the  particular 
arrestment  wire  engaged .  Such  evidence  suggests  that  reliable  information  can 
be  obtained  through  the  use  of  such  subjective  indices. 


Developing  a  criterion  for  pilot  performance  across  all  aircraft  communi¬ 
ties  is  a  formidable  task  due  to  differences  in  mission  orientation.  For  this  rea¬ 
son  it  was  decided  to  limit  the  scope  of  the  present  research  effort  to  only  one 
group — the  F-4  fighter  community .  The  RAG  seemed  to  be  a  fertile  area  for 
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investigation  due  to  its  similarity  to  actual  fleet  operations.  It  was  reasoned  that 
the  skills  required  for  the  successful  completion  of  this  phase  of  training  should 
best  reflect  those  required  in  the  fleet . 


RAG  training  in  the  F-4  aircraft  is  broken  down  into  a  series  of  stages. 
Within  each  stage,  the  replacement  pilot  is  expected  to  demonstrate  proficiency 
in  the  performance  of  a  wide  variety  of  skills  and  procedures .  It  is  from  the  rat¬ 
ings  which  he  receives  on  each  of  these  items  that  his  stage  grades  are  deter¬ 
mined.  It  seems  reasonable  that  within  each  stage,  certain  of  these  skills  and 
procedures  should  be  of  greater  importance  than  others .  It  should  be  possible 
to  isolate  a  small  set  of  "critical"  items  which  would  discriminate  among  replace¬ 
ment  pilots  of  differing  ability  levels .  Such  skills ,  it  is  contended ,  should  form 
the  basis  upon  which  an  adequate  fleet  performance  measure  might  be  con¬ 
structed  . 


Shannon,  Waag,  and  Ferguson  (3)  attempted  such  an  item  analysis  for 
each  of  the  stages  of  F-4  training  at  VF-101 ,  the  East  Coast  RAG  squadron .  For 
each  of  the  stages  analyzed ,  a  small  set  of  graded  items  was  selected  on  the 
basis  that  they  could  adequately  discriminate  among  replacement  pilots  according 
to  their  final  RAG  grade .  A  series  of  regression  analyses  revealed  these  iso¬ 
lated  items  to  be  highly  predictive  of  both  the  stage  grades  from  which  they  were 
obtained  and  the  final  RAG  grade .  The  usefulness  of  such  an  approach  is 
demonstrated  in  a  later  study  by  Shannon  and  Waag  (4)  in  which  these  isolated 
items  were  found  to  be  highly  predictive  of  accidents/incidents  despite  the  fact 
that  the  assigned  grades  were  not . 


The  authors  contend  that  such  "critical"  items  isolated  in  this  manner 
should  form  the  basis  from  which  an  adequate  measure  of  fleet  performance  might 
be  developed .  The  purpose  of  the  present  investigation  was  to  replicate  the 
findings  of  the  previous  investigation  with  data  obtained  from  VF-121,  the  F-4 
West  Coast  RAG  squadron .  In  the  event  similar  results  were  obtained ,  strong 
support  would  be  provided  to  the  approach  undertaken . 


PROCEDURE 


The  sample  group  consisted  of  59  replacement  pilots  assigned  to  VF-121 , 
the  West  Coast  squadron  for  RAG  training  in  the  F-4  aircraft  between  June  1970 
and  March  1972.  During  this  period  the  training  syllabus  consisted  of  the 
following  stages:  Familiarization  (PF);  Instruments  (PI);  Weapons  Systems 
(PS);  Conventional  Weapons  (PW);  Tactics  (PT);  and  Field  Mirror  Landing 
Practice/Carrier  Qualification  (FMLP/CQ) .  For  22  of  the  pilots,  no  CQ  data  were 
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available  due  to  the  fact  that  this  stage  had  just  recently  been  eliminated  from  the 
training  syllabus .  Student  flight  records  were  obtained  and  item  analyses  per¬ 
formed  for  each  of  these  stages . 


The  grading  system  for  the  West  Coast  RAG  requires  the  instructor  to 
rate  the  replacement  pilot’s  performance  on  each  skill  and  procedure  as  falling 
within  one  of  five  categories:  Above  Average  (AA) ;  Average  (A) ;  Below  Average 
(BA) ;  Marginal  (M) ;  and  Unsatisfactory  (U) .  For  each  item ,  a  weighted  score 
was  derived  reflecting  the  distribution  of  AA's,  BA's,  M's,  and  U's.  For  each 
item  within  a  stage,  the  number  of  AA's,  BA's,  M's,  and  U's  were  recorded  for 
each  replacement  pilot  across  all  hops .  The  number  of  M's  and  U's  were  com¬ 
bined  to  form  a  single  category .  The  resulting  sums  were  then  weighted  by  +1 , 
-1,  and  -2  respectively.  These  weighted  totals  were  then  summed  in  order  to 
yield  an  overall  score.  The  resulting  item  measures  were  then  standardized  and 
transformed  to  T  scores .  These  derived  scores  for  each  of  the  items  were  then 
related  to  the  individual  stage  grade  as  well  as  the  final  RAG  grade  in  a  series  of 
correlational  and  regression  analyses.  Only  those  items  were  included  which 
had  been  selected  in  the  previous  investigation . 


RESULTS 


Of  the  17  items  initially  selected  from  the  East  Coast  sample  (3)  14  were 
found  to  be  graded  items  for  the  West  Coast  sample.  For  the  PF  stage,  four  items 
were  found  to  be  common.  These  included  Headwork,  Basic  Airwork, 
Maneuvers,  and  VFR  Glide  Slope  Control.  For  the  PS  stage,  three  items  were 
common,  including  Headwork,  Basic  Airwork,  and  Altitude  Control.  For  the  PT 
stage,  four  items  were  the  Eame.  These  included  Basic  Airwork,  Aggressive¬ 
ness,  Offensive  Air  Combat  Maneuvering,  and  Lookout  Doctrine.  For  the 
FMLP/CQ  stages,  three  items  were  common,  including  Glide  Slope  Control, 

Speed  Control,  and  Power  /Nose  Control.  In  the  previous  investigation,  none 
of  the  PI  items  discriminated  among  the  categories  of  replacement  pilots  while 
the  PW  data  were  not  available.  Consequently,  no  items  were  included  for  these 
stages . 


Correlations  were  computed  between  the  derived  score  for  each  of  the  14 
items  and  the  stage  grade  from  which  they  were  obtained,  as  well  as  the  final 
RAG  grade.  These  results  are  summarized  for  both  coasts  in  Table  I .  Using 
the  final  RAG  grade  as  a  criterion,  a  multiple  correlation  of  .852  was  obtained 
for  the  East  Coast  sample  when  all  14  items  were  entered  into  the  prediction 
equation.  Using  the  beta  weights  derived  from  the  resulting  equation,  predic¬ 
tions  were  derived  for  pilots  in  the  West  Coast  sample.  The  resulting  correlation 
between  predicted  and  observed  RAG  grades  was  .776.  A  regression  analysis 
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Table  I 


SUMMARY  OF  ZERO-ORDER  CORRELATIONS  BETWEEN 
ITEM  SCORES  AND  RAG  GRADES 


Correlations  With: 

Item  Stage  Stage  Grade  Total  Grade 


East 

West 

East 

West 

Head  work 

PF 

.510 

.322 

.035 

.088 

Basic  Airwork 

PF 

.699 

.526 

.276 

.250 

VFR  G/S  Control 

PF 

.653 

.733 

.244 

.746 

Maneuvers 

PF 

.459 

.325 

.108 

.200 

Altitude  Control 

PS 

.434 

.513 

.302 

.434 

Headwork 

PS 

.291 

.256 

.121 

.240 

Basic  Airwork 

PS 

.255 

.343 

.255 

.279 

Aggressiveness 

PT 

.731 

.554 

.447 

.540 

Offensive  ACM 

PT 

.867 

.663 

.440 

.649 

Lookout  Doctrine 

PT 

.618 

.642 

.265 

.724 

Basic  Airwork 

PT 

.639 

.439 

.410 

.478 

Speed  Control 

FMLP/CQ 

.579 

.194 

.611 

.087 

Glide  Slope  Control 

FMLP/CQ 

.645 

.164 

.549 

.320 

Power /Nose  Control 

FMLP/CQ 

.695 

.130 

.646 

.159 

using  beta  weights  derived  from  the  West  Coast  sample  data  yielded  a  mul¬ 
tiple  correlation  of  .956.  In  all  cases  the  multiple  R's  were  highly  signifi¬ 
cant. 


DISCUSSION 


The  results  of  this  investigation  confirm  previous  findings  that  a  small 
set  of  procedures  and  skills  can  be  isolated  which  are  highly  predictive  of 
flight  ability  as  estimated  by  the  final  RAG  grade.  As  the  data  indicate,  those 
items  selected  in  the  East  Coast  sample  were  also  found  to  be  highly  predictive 
of  performance  for  the  West  Coast  sample.  However,  some  differences  did 
emerge  across  the  two  squadrons.  The  major  disagreement  appears  to  be  in  the 
relationship  of  the  FMLP/CQ  items  to  the  criterion.  This  may  in  part  result  from 
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the  fact  that  carrier  qualifications  had  been  eliminated  from  a  sizeable  percentage 
of  the  pilots  within  the  sample.  As  a  result,  there  were  instances  in  which  only 
the  FMLP  data  were  included . 


Despite  these  discrepancies,  it  is  highly  encouraging  to  find  that 
reliable  predictions  can  be  obtained  from  only  a  small  subset  of  the  original  item 
pool.  In  fact  these  14  items  represent  less  than  15%  of  the  total  set  which  instruc¬ 
tors  are  required  to  rate .  It  is  suggested  that  these  represent  the  most  impor¬ 
tant  skills  within  RAG  training ,  and  as  a  result  of  their  similarity  to  fleet  opera¬ 
tions,  should  be  the  most  reflective  of  fleet  performance.  On  the  basis  of  these 
findings  a  rating  form  was  developed  for  the  purpose  of  assessing  such  perfor¬ 
mance  in  fleet  squadrons.  This  rating  form,  which  is  contained  in  Appendix  A, 
not  only  contained  the  14  items  common  to  both  East  and  West  Coasts,  but  also  a 
restricted  number  of  items  which  were  highly  predictive  of  performance  in 
either  one  of  the  RAG  squadrons .  In  so  doing  it  was  hoped  that  the  rating  form 
would  contain  all  possible  skill  elements  of  F-4  pilot  performance.  The  useful¬ 
ness  and  validity  of  this  form  is  currently  under  investigation. 
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F-4  FLEET  PERFORMANCE  RATING  FORM 


This  form  is  to  be  used  by  all  F-4  West  Coast  Squadrons  in  evaluating 
the  performance  of  certain  selected  pilots .  The  undergraduate  and  RAG  train¬ 
ing  jackets  of  these  pilots  have  been  thoroughly  examined .  Now  all  that 
remains  is  a  fleet  evaluation  of  these  pilots.  Hopefully,  this  will  provide  a 
reliable  criterion  measure  against  which  both  graduate  and  undergraduate  pilot 
performance  can  be  measured.  Completed  forms  will  be  used  as  research  data 
only  and  will  have  no  influence  on  the  careers  of  the  individuals  being  rated . 

This  form  was  developed  by  analyzing  the  RAG  training  jackets  of 
replacement  pilots  in  VF-101  and  VF-121.  Items  within  each  stage  of  training 
were  included  on  this  form  which  were  highly  predictive  of  the  final  RAG  grade , 
and  consequently  the  best  discriminators  among  replacement  pilot's  flight 
ability . 

This  form  should  be  completed  by  the  Squadron  Commanding  Officer,  or 
a  designated  individual  who  is  best  knowledgeable  concerning  a  particular 
pilot's  flight  performance.  There  are  17  items  to  be  rated  on  a  7-point  scale. 
This  scale  is  based  on  a  pilot's  relative  standing  among  other  F-4  pilots  through¬ 
out  the  entire  community.  The  percentages  listed  beneath  the  scale  points  indi¬ 
cate  what  proportion  of  pilots  should  fall  within  each  of  these  categories  for  the 
entire  community.  The  pilots  you  will  be  asked  to  rate  will  not  necessarily  con¬ 
form  to  these  percentages.  Clearly,  not  everyone  can  be  outstanding  and  rated 
in  the  top  5%  of  his  group.  Likewise,  it  is  possible  for  an  individual  to  be 
within  the  bottom  5%  of  his  group  and  still  be  an  acceptable  pilot  with  respect  to 
flying  ability. 

Additional  comments  concerning  performance,  flight  mishaps,  or  other 
flight  difficulties  should  be  noted  in  the  space  provided .  Space  is  also  provided 
for  suggestions/comments  which  are  pertinent  to  the  content  and  construction  of 
the  rating  form  itself. 

Completed  forms  should  be  mailed  in  the  pre-addressed  envelopes  to: 
Officer  in  Charge,  Code  L5,  Naval  Aerospace  Medical  Research  Laboratory, 

Naval  Aerospace  Medical  Institute,  Naval  Aerospace  and  Regional  Medical 
Center,  Pensacola,  Florida,  32512. 


Are  there  any  suggestions /comments  concerning  the  content  or  the  construction 
of  this  rating  form?  _ 


A-l 
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NAME 


SQUADRON 


FILE  NUMBEF 


Relative  Position  Compared 
With  Other  F4  Pilots 


Acceptable 

Good 

Outstanding 

1 

2 

3 

4 

5 

6 

7 

5% 

10% 

20% 

30% 

20% 

10% 

5% 

A.  How  well  does  this  [jilot  rate  on  the  items  listed 
below  compared  with  other  pilots  in  the  F4 
community? 

1.  General 

A.  Basic  Airwork/Aircraft  Handling  X  X 

B.  Head  work  X  X 

C.  Formation  (inflight  refueling,  tactical, 

enroute,  takeoff,  landing)  X  X 

D.  Tactical  Aggressiveness  X  X 

E.  Crew  Coordination  (commentary,  team¬ 
work,  command  response)  X  X 

2.  Landings-Carrier/Field 

A.  Glide  Slope  Control  (power/nose  con¬ 
trol,  speed  control,  scan,  line-up)  X  X 

B.  Landing/Landing  Roll-Out  X  X 

3.  Instruments 

A.  Departure  X  X 

B.  Penetration/Approach  X  X 

4.  Conventional  Weapons 

A.  Bomb  Pattem/Dive  Parameters  (dive 

angle,  roll-in  airspeed,  release,  pull-out)  X  X 

B,  Overall  Accuracy  X  X 

5.  Weapons  Systems 

A.  Flying  Technique  (altitude/heading/ 

airspeed  control)  X  X 

B.  Radar  Intercept  X  X 

6.  Tactics 

A.  Offensive  ACM  X  X 

B.  Defensive  ACM  X  X 

C.  Lookout  Doctrine  X  X 

D.  Fuel  Management  X  X 


(Cfi-cle  one  “X”  for  each  item) 


X 

X 

X 
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B.  Has  this  pilot  had  any  accidents,  incidents,  boards,  or  had  his  “wings”  pulled? _ 

C  Are  there  any  additional  comments  concerning  this  pilot’s  ability,  mental  attitude,  motivation,  or  any  other 
pertinent  information? 
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